Evaluating the Federal Innovative Assessment Demonstration Authority: Early Implementation and Progress of State Efforts to Develop New Statewide Academic Assessments. Evaluation Report. NCEE 2023-004

Education officials have long hoped that the statewide academic assessments most students take each year could be used not only for accountability but also to guide instruction. Congress established the Innovative Assessment Demonstration Authority (IADA) program in 2015 to help address this goal, offering up to seven states temporary flexibility from federal testing requirements so that they may more easily make progress toward replacing their current assessments with more innovative ones. The key incentive to participate in IADA is that students trying out the innovative assessment are not required to also take the state's current assessment. However, states approved for IADA must still show that their innovative assessments meet most requirements for federal accountability, and they are expected to implement the new assessments statewide within 5 years. This report describes the progress of the first five assessment systems approved under IADA in order to help policymakers consider expanding the program to more states. The report is primarily based on an analysis of states' IADA applications and performance reports to the U.S. Department of Education through the 2020-2021 school year and is part of a broader evaluation of IADA required by Congress. [For the Appendix, see ED627873. For the Study Highlights, see ED627880.]

Towards Augmented Reality Guiding Systems: An Engineering Design of an Immersive System for Complex 3D Printing Repair Process

Over the past decade, additive manufacturing (AM) has become widely adopted for both prototyping and, more recently, end-use products. In particular, fused deposition modeling (FDM) is the most widespread form of additive manufacturing due to its low cost, ease of use, and versatility. While additive processes are relatively automated, many steps in their operation and repair require trained human operators. Finding such operators can be difficult, as highlighted during the recent COVID-19 pandemic. Augmented reality (AR) systems could significantly help address this challenge by automating the training for 3D printer operators. Given multidimensional design choices, however, a research gap exists in the system requirements for such immersive guidance. To address this need, we explore the applicability of AR to guide users through a repair process. In that context, we report on the system design as well as the results of the AR system assessment in a qualitative study with experts. © 2023 IEEE.

Improvement of the Quality of Question Papers for Online Examinations Toward Simultaneous Enhancement of Students', Learning

The present study is related to the problem associated with student assessment with online examinations at higher educational institutes (HEIs). With the current COVID-19 outbreak, the majority of educational institutes are conducting online examinations to assess their students, where there would always be a chance that the students go for malpractice. It is difficult to set a question paper for any technical course with great novelty. Under these circumstances, safeguarding academic integrity has become a challenge for HEIs. This study is aimed at improving the quality of questions for online exams to increase the accountability of HEIs by proper evaluation of their students. A detailed procedure with suggestions for setting the questions for technical courses, in the format of assertion and reason, matching, multiple select types, etc., has been discussed with adequate examples. It deals with a strategy for ensuring that all the students are held to the standards that are reflected in their grades. The proposed evaluation method has been implemented on a test batch and presented the results along with a comparison with that of traditional question papers. It is witnessed that there is a simultaneous enhancement of students'learning as an additional benefit of implementing the proposed learning-oriented assessment method. IEEE

Evaluating the Federal Innovative Assessment Demonstration Authority: Early Implementation and Progress of State Efforts to Develop New Statewide Academic Assessments. Study Highlights. NCEE 2023-004h

Education officials have long hoped that the statewide academic assessments most students take each year could be used not only for accountability but also to guide instruction. Congress established the Innovative Assessment Demonstration Authority (IADA) program in 2015 to help address this goal, offering up to seven states flexibility from federal requirements so that they may more easily make progress toward replacing their current assessments with more innovative ones. The report describes the early progress of the first five IADA systems to help policymakers consider expanding the program to more states. The report is primarily based on analyses of states' IADA applications and annual performance reports through the 2020-2021 school year and is part of a broader evaluation of IADA required by Congress. This Study Highlights describes the key findings from the report. [For the full report, see ED627872. For the Appendix, see ED627873.]

Research Progress of Antiviral Coatings

With the large-scale spread of COVID-19 around the world, it has caused serious damage to the health of people around the world. In addition to being transmitted by various droplets, viruses can also be transmitted by human touch of contaminated surfaces. However, as a commonly used surface antiviral method, disinfectants have the disadvantage of discontinuously inactivating viruses, which is bad for inhibiting the spread of various infectious viruses. Therefore, it is urgent to protect the surface of daily objects from virus pollution to eliminate the spread of various respiratory viruses ( such as Corona Virus Disease 2019, SARS-CoV-2). From this point of view, it is very important to design and develop effective antiviral coatings. This paper discusses the working mechanisms, performance evaluation methods, processing technologies, practical applications and research progress of nanoparticle antiviral coatings and polymer antiviral coatings for SARS-CoV-2, and also proposes some strategies to design more effective antiviral coatings from the perspective of different types of antiviral coatings. Although some of these antiviral coatings are still in the experimental stage, they still show great potential in the antiviral field.

USABILITY TESTING OF INTERACTIVE SYSTEMS CONDUCTED BY UNIVERSITY STUDENTS DURING A PANDEMIC

This case study describes a usability testing course in which students learn by practicing several evaluation methods. The on-campus format makes it possible for teachers and students to meet to discuss recorded test sessions and students can observe other students' execution of pilot studies conducted on campus. The COVID-19 pandemic placed new demands on this course. In-person activities were avoided by some students and many test participants. Some student teams tried remote usability testing. Interestingly, screen recordings (with sound) of the test sessions show that remote testing sometimes helped the students focus more on observation and less on (inappropriately) guiding the test subjects. Another effect was that the students found it easier to recruit participants than during the previous years when the university was teeming with students, lecturers, and non-academic staff. However, the recruited participants were often notably limited to the students' circles of friends. © (2022) by Association for Information Systems (AIS) All rights reserved.

Principles for evaluation of one health surveillance: The EVA book

This book outlines essential elements of the evaluation of health surveillance within the One Health concept. It provides an introduction to basic theoretical notions of evaluation and vividly discusses related challenges. Expert authors cover the entire spectrum of available, innovative methods, from those for system process evaluations to methods for the economic evaluation of the surveillance strategies. Each chapter provides a detailed description of the methodology required and the tools available as illustrated by practical examples of animal health or One Health surveillance evaluations in both developed and developing countries. Targeting not only scientists, including epidemiologists, but also technical advisers of decision-makers, the present work is suitable for the evaluation of any type of health surveillance system - animal, human or combined - regardless of the socio-economic context. The volume is richly equipped with practical tools and examples, which enables the reader to apply the methods described. Increasing importance of health surveillance, and threats from disease outbreaks such as the coronavirus pandemic, underline the practical relevance of this work, which will fill an important gap in the literature. © Springer Nature Switzerland AG 2022. All rights reserved.

The Effectiveness of Virtual Assessment Center Platform

The Assessment Center (AC) Method has been widely used as one of the evaluation methods for selecting, promoting, and developing leaders in a company. In Indonesia, AC has also been used for selecting the Board of Directors, especially in State-Owned Enterprises (Peraturan Menteri BUMN, 2021). In the digital era, the demand for a fast and flexible assessment center increased and became essential for selecting the right candidate as evidenced by a 57-percent rise of the Assessment Center project from 2017 to 2021. In response to COVID-19 pandemic as well as to cope with the increasing demand, Assessment Center Indonesia (ACI), by Telkom Indonesia, has been developing a Virtual Assessment Center (VAC) which has been implemented for AC in Telkom since the middle of 2020. The current study was conducted as an effort to examine its effectiveness. ACI tested the effectiveness of Virtual Assessment Center (VAC) using the effectiveness framework that measures: 1) ease of use, 2) perceived usefulness, and 3) the consistency of virtual and offline AC results. The first and second aspect of effectiveness indicators were collected by using questionnaires and was analyzed using descriptive statistics. As for testing the consistency of virtual and offline AC result, the ratings of nine competencies from 32 employees who took offline and virtual AC in 2021-2022 were gathered and were analyzed using the Wilcoxon Signed Rank Test. The result showed that participants' perceived ease of use in the Virtual AC group is slightly higher than the offline method. As for the human capital users' usefulness, it was confirmed that the Virtual AC was perceived as more effective than offline methods. Moreover, from the Virtual AC administrator's perspective, they agreed that the Virtual AC supported the flexibility of time, place, and number of participants. In addition, Wilcoxon Signed Rank Test also validated that there were no significant differences between the pairs' rating of nine competencies measured by either offline or virtual methods (p> 0.05). The results from the three perspectives above showed that Virtual AC can be regarded as an effective replacement of the offline assessment. Further analysis is still needed to compare the validity of offline versus virtual AC. Practical implications also provided in the research paper. © 2022 IEEE.

Use of a multi-method approach to rapidly assess the impact of public health policies at the state and local level: a case study of flavored e-cigarette policies.

BACKGROUND: E-cigarettes are the most-commonly used tobacco product by youth since 2014. To prevent youth access and use of e-cigarettes, many U.S. states and localities have enacted policies over a relatively short period of time. The adoption of these policies has necessitated timely data collection to evaluate impacts. METHODS: To assess the impact of flavored e-cigarette policies in select states and local jurisdictions across the United States, a multi-method, complementary approach was implemented from July 2019 to present, which includes analyses of cross-sectional online surveys of young people ages 13-24 years with retail sales data. RESULTS: From February 2020 through February 2023, cross-sectional surveys have been conducted in three cities, one county, and eight states where policy changes have been enacted or are likely to be enacted. Data collection occurred every six months to provide near real-time data and examine trends over time. Additionally, weekly retail sales data were aggregated to showcase monthly sales trends at the national level and for the selected states. DISCUSSION: This rapid and efficient method of coupling online survey data with retail sales data provides a timely and effective approach for monitoring a quickly changing tobacco product landscape, particularly for states and localities where rapidly-available data is often not available. This approach can also be used to monitor other health behaviors and relevant policy impacts.

Study on Exercise Evaluation based on Pose Estimation for Remote Exercise System

The global impact of COVID-19, the disease caused by the novel coronavirus, has caused the whole world to quarantine. Since most of the gyms are closed during the pandemic, people are opting to exercise from home. The goal of this research is to combine video chat application and pose estimation function to design a multiuser fitness website that can increase the competition or cooperation level, which in turn increases the motivation to continue exercise in a remote fitness setting. This paper presents an evaluation method of exercise with "MoveNet"model and describes the design of the remote fitness application using that method. As a result, in our system, we have implemented four different exercises such as "Jumping Jacks,""Squat,""Pushup,"and "Curls."Our system allows the evaluation of those exercises and can design remote exercise applications using this algorithm. © 2022 ACM.

Measuring Google Classroom's Usability Level from an Expert Perspective

The demand for the use of distance learning media, especially at the beginning of the covid-19 pandemic, led to the high use of one of the long-known distance learning media, namely Google Classroom. This study aims to provide empirical evidence of the effect of perceived usability on the use of Google Classroom recently. We use heuristic evaluation method to evaluate user interface of e-Learning with four expert participants who are also instructor/teacher in some courses, the data is presented in form of quantitative and in-depth in the form of qualitative. Result of the study showed that the highest severity level was 1 (cosmetic usability problem) in the first heuristic (visibility of system status): 2 from 4 expert said that Google Classroom does not provide visuals that help the user understand in which screen they are standing;and sixth heuristic (recognition rather than recall): 2 from 4 expert said that Google Classroom does not indicate which controls are currently inactive in clear visual way. This study introduces a new approach, risk opportunity measurement which is based on risk-based thinking of ISO 9001:2015 (standard of quality management system), to provide recommendations for improvement of problems found by evaluators to stakeholders based on principles/heuristics that have the highest to lowest risk opportunities for improvement and further development. © 2022 IEEE.

An evaluation of a rapid conversion to teleSANE in response to COVID-19.

Sexual Assault Nurse Examiners (SANEs) provide expert, comprehensive medical forensic care to patients who present for services following a sexual assault. Because SANEs are not consistently available, telehealth technology is being explored as a means to provide access to this expert care (i.e., teleSANE). During the COVID-19 pandemic, teleSANE offered additional potential benefits by reducing the length of time spent and number of providers in patient exam rooms, the need for personal protective equipment that was in high demand and short supply, and provider anxiety related to providing in-person care. In the summer of 2020, the Massachusetts SANE program rapidly and temporarily converted five hospitals from in-person SANE care to teleSANE. An evaluation team interviewed 23 providers using a rapid research and evaluation methods approach to assess the temporary model and inform the future of SANE care. Evaluation findings reveal it is possible to rapidly and temporarily convert hospitals from in-person to teleSANE care in a time of broad uncertainty, and that such a change requires intensive and thoughtful planning; a shared commitment to being supportive, flexible, and responsive; and specific experience and expertise. Considerations for communities exploring how best to ensure consistent, equitable access to SANEs are discussed.

Every Tool in the Toolbox: Pursuing Multilevel Institutional Change in the DeLTA Project

For decades, educators and policy makers have called for reform in higher education, yet now the urgency is palpable. The COVID-19 pandemic and heightened attention to systemic racism have highlighted the fact that outdated teaching practices can stunt student learning and trust of science, maintain systemic biases, and prevent equitable education. Promoting change to outdated teaching practices requires fundamental shifts at each level of a university--among faculty, departments, and the institution as a whole. The Departmental and Leadership Teams for Action (DeLTA) project at the University of Georgia pursues transformative shifts in policies and practices related to undergraduate science, technology, engineering, and math (STEM) education. This article provides examples of how DeLTA applies various change perspectives to shift thinking, practices, and policies related to evaluating teaching.

English Language Arts Instructional Systems in the First Full Year of COVID-19. Research Report. RR-A279-2

Since March 2020, the coronavirus disease 2019 (COVID-19) pandemic has placed unprecedented stresses on the public education system in the United States. At every level, from the U.S. Department of Education down through local districts and individual schools, the pandemic has presented formidable challenges. Many of these challenges have been operational in nature but there have also been substantial instructional challenges during the pandemic. Early insight into the effects of the COVID-19 pandemic on public schooling identified the challenges that schools faced in the pivot to online learning in spring 2020. Large proportions of teachers reported through the RAND American Teacher Panel (ATP) that they had not received adequate guidance from their school systems to serve particular populations of students, such as students with disabilities (SWDs), homeless students, and English learners (ELs). These pandemic-era instructional challenges compound existing barriers to quality instruction. This report examines issues of instructional system coherence during the 2020-2021 school year. Specifically, the report investigates teachers' perceptions of: (1) guidance they received about ELA instruction, (2) guidance around addressing the needs of traditionally underserved students, (3) coherence of their ELA instructional system, and (4) presence of contextual conditions identified through literature as supporting coherence.

Teachers' Experience in E-Assessment: Case Study of EFL Teachers in Algerian Universities

In the wake of the Corona Virus pandemic, many universities have rapidly shifted to remote education. In this regard, students are enrolled in online learning instead of, or sometimes along with, physical classroom presence. As a result, the online assessment also appeared as an alternative method to assess students. The current research paper aims to provide an in-depth look at the online reviews in EFL teaching and learning in the Algerian context during the COVID-19 pandemic. The main questions that set the study are: How do Algerian English language teachers perceive e-assessment via Moodle? What impact did the online reviews have on assessment practices?, How did an online assessment affect students 'achievements? To answer these questions, the researcher conducted a study with Algerian EFL teachers and students from three Universities. The researcher collected data through a questionnaire addressed to EFL teachers and a comparative analysis of Project-Based Language essays done by the students in face-to-face assessment and online assessment. The findings revealed that although the challenges facing teachers in online review, positive attitudes appeared towards this method. The analysis also demonstrated that the assessment practice used mainly by participated teachers is a summative assessment. The results also showed a big difference in students' achievements between face-to-face assignments and online evaluation, in favor of e-assessment.

A Framework to Implement Academic Digital Badges When Reskilling the IT Workforce

Information technology (IT) plays an increasingly significant role in today's world. The COVID-19 pandemic has increased that reliance. Employers in all industries are struggling with considerable shortages of skilled IT workers and are seeking innovative alternatives to meet these needs. Digital badging and micro-credentials have emerged as an alternative system to validate skills. The issuing of digital badges has spanned across a wide spectrum of settings and purposes. However, there is currently less application of digital badges in graduate-level programs to link to important academic or subject-matter related achievements and higher-level competencies. Reskilling and upskilling existing employees need a more practical and sustainable approach and often do not require completion of an entire IT master's degree. To align with such demand, this study aims to demonstrate how a digital badging system can be used to validate stackable certificates for micro-credentials in a graduate-level program. The paper starts with a background introduction of the current IT employment landscape. The next two sections provide an overview of micro-credentials in workforce development and higher education today. This is followed by a section on our conceptual framework used to determine the potential value of digital badging to our graduate programs. The next section focuses on the case study. The last sections conclude with the lessons learned and the future research directions.

The value of varying question design

While some tertiary mathematics educators approximated the familiar invigilated, closed-book assessment regime in the online environment forced upon us all by a pandemic, others either by choice or necessity needed to devise a new way to assess their students' learning. This Classroom Note both synthesizes advice from the literature about what might comprise 'internet resistant' question design and provides practical, specific examples to demonstrate how such advice can be put into everyday practice. The examples are annotated, and the potential long-term utility of such question design is discussed, whatever the future of assessment may be.

Academics' Perception and Practices of Online Education during the COVID-19 Pandemic: The Case of Turkey

COVID-19 affected higher educational institutions around the world. This study focused on academics' perceptions and practices of online education based on their academic discipline and previous online education experience. The survey research design used and an e-survey was conducted in May 2020, during which the lockdown restrictions were rather tight in Turkey. The analyzes were carried out on the responses of 8,242 academics' using the omnibus chi-square test and t-tests. The results point out that academics' time allocation to preparation for courses, student counselling, and lectures are varying due to their academic disciplines. Generally, academics have a negative perception of online education though there is a difference between the experienced ones and others. In particular, academics with online education experience exhibited lower negative perceptions of online education than those lacking such experience. Maybe one of the most problematic parts of this transition process for academics is how to ensure the reliability of exams. Our results showed that even the academics with online education experience have a higher level of distrust towards educational measurement and evaluation methods.

Do the Scores Obtained from Online Applications Correspond to Face-to-Face Exam Scores?

The emergent COVID-19 pandemic in Wuhan, China, at the end of 2019 has affected the whole world in a short time. Therefore, schools have been closed worldwide, and online learning opportunities have been exploited. Although lessons are taught online, problems have been experienced about measuring and evaluating lessons. This quasi-experimental study with a quantitative method was conducted to offer suggestions for the measurement problems. The present study examined the relationship between the measures of academic achievement obtained through different approaches from the students studying at a university. According to the results, the achievement points earned by the students through weekly blogs had a high level of correlation with the traditional final scores. However, the scores of the practices such as online exams and term papers were not an acceptable fit with the final exams. Therefore, the scores of online missions extended over time can be claimed to be the approach that best substitutes the final exams.

How COVID-19 Has Changed Language Assessments in Higher Education: A Practitioners' View

This chapter examines the responses of Higher Education Institutions (HEIs) to assessment-related challenges introduced by the COVID-19 pandemic. The sudden move to online teaching and assessments required institutions to re-think their approaches to current practices and provide alternative solutions almost overnight. Our survey aimed to gather the perceptions of language teaching practitioners on how the languages sector dealt with the move to digital assessments, with particular emphasis on the delivery of online assessments, the impact on different language skills, as well as academic integrity. Results show that, whilst different institutions adopted varying practices, they seem to share similar concerns in addressing the fast-changing environment and ensure that academic integrity remains at the forefront in the move to the digital world. [For the complete volume, "Innovative Language Teaching and Learning at University: Facilitating Transition from and to Higher Education," see ED619814.]